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(54) Digital watermarking and methods for security documents 



(57) Security documents (e.g. passports, cun-ency. 
event tickets, and the like) are encoded to convey 
machine-readable multi-bit binary information (e.g. dig- 
ital watermark), usually in a manner not alerting human 
viewers that such information is present. The docu- 
ments can be provided with overt or subliminal calibra- 
tion patterns. When a document incorporating such a 
pattern Is scanned (e.g. by a photocopier), the pattern 
facilitates detection of the encoded information notwith- 
standing possible sealing or rotation of the scan data. 
The calibration pattern can serve as a canrier for the 
watermark information, or the watermark can be 
encoded independently In one embodiment, the water- 
mark and the calibration pattern are formed on the doc- 
ument by an intaglio process, with or without ink. A 
photocopier responsive to such markings can take pre- 
determined action if reproduction of a security docu- 
ment is attempted. A passport processing station 
responsive to such markings can use the decoded 
binary data to access a database having information 
concerning the passport holder Some such appara- 
tuses detect both the watermark data and the presence 
of a visible structure characteristic of a security docu- 
ment (e.g., a printed seat of the document's issuer). 



FIG. 12 




CM 
< 
CO 



oo 
o 

CL 
LU 



Primed b/ Xerox (UK) Busmess Services 
2.16 7/36 



BNSDOCID: <EP 0981113A2 I > 



1 



EP0 981 113 A2 



Description 

Related Application Data 

[0001 J This application is related to commonly-owned 
patents 5.862.260. 5.850.481. 5.841,978. 5.841.886 
5.832.119, 5,822,436. 5.809.160, 5,745.604. 
5.636.292. 5.710.834, 5,721,788. 5.748,763. 
5,748,783. and 5.768.426. 

Field of the Invention 

[0002] The present invention relates to methods and 
systems for inconspicuously embedding binary data in 
security documents, and associated methods/systems 
for detecting/decoding such data, ("Security document" 
is used herein to refer to negotiable financial instru- 
ments (e.g. banknotes, travelers checks, bearer bonds), 
passports, visas, other immigration documents, stock 
certificates, postal stanps, lottery tickets, sports/con- 
cert tickets, etc.) One application of this the invention is 
in discouraging counterfeiting of security documents. 
Another is in transferring machine-readable information 
through such documents, without alerting human view- 
ers to the presence of such information. 

Backoround and Summary of the Invention 

[0003] Digital watermarking {sometimes termed "data 
hiding" or "data emljedding") is a growing field of 
endeavor, with several different approaches. The 
present assignee's work is reflected in the patents and 
applications detailed above, together with laid-open 
PCT application W097/43736. Other work is illustrated 
by U.S. patents 5.734,752, 5.646.997, 5.659,726. 
5.664.018, 5.671,277. 5.687.191. 5,687.236. 
5.689,587. 5.568.570, 5.572,247, 5.574,962. 
5,579.124. 5,581.500. 5,613.004. 5,629,770. 
5,461,426, 5,743.631, 5.488,664. 5.530.759, 
5.539.735. 4.943.973. 5,337.361, 5,404.160, 
5.404,377. 5.315.098, 5,319.735, 5,337.362, 
4.972.471. 5,161.210, 5,243.423. 5,091,966. 
5,113.437, 4,939.515. 5.374.976. 4.855.827, 
4,876.617. 4,939.515, 4.963.998. 4.969,041, and pub- 
lished foreign applications WO 98/02864. EP 822,550, 
WO 97/39410. WO 96/36163, GB 2.196.167. EP 
777,197. EP 736,860. EP 705.025. EP 766.468, EP 
782,322, WO 95/20291 , WO 96/26494. WO 96/36935. 
WO 96/42151, WO 97/22206. WO 97/26733. Some of 
the foregoing patents relate to visible watermarking 
techniques. Other visible watermarking techniques (e.g. 
data glyphs) are described in U.S. patents 5.706.364. 
5.689.620. 5.684,885. 5.680.223. 5.668.636. 
5.640.647. 5.594.809. 

[0004] Much of the work in data embedding is not in 
the patent literature but rather is published in technical 
articles. In addition to the patentees of the foregoing 
patents, some of the other workers In this field (whose 



watermark-related writings can by found by an author 
search in the INSPEC or NEXIS databases, among oth- 
ers) include I. Pitas, Eckhard Koch, Jian Zhao. 
Norishige Morimoto, Laurence Boney, Kineo Matsui. 
5 A.Z. Tirkel. Fred Mintzer. B. Macq. Ahmed H. Tewfik, 
Frederic Jordan, Naohisa Komatsu. Joseph O'Rua- 
naidh. Neil Johnson. Ingemar Cox. Minerva Yeung. and 
Lawrence O'Gorman. 

[0005] The artisan is assumed to be familiar with the 
10 foregoing prior art. 

[0006] In the following disclosure it should be under- 
stood that references to watermarking encompass not 
only the assignee's watermarking technology, but can 
likewise be practiced with any other watermarking tech- 
15 nology, such as those indicated aljove. 

[0007] Watermarking can be applied to myriad forms 
of information. The present disclosure focuses on its 
applications to security documents. However, it should 
be recognized that the principles discussed below can 
so also be applied outside this area 

[0008] Most of the prior art in image watermarking has 
focused on pixelated imagery (e.g. bit-mapped images. 
JPEG/MPEG imagery. VGA/SVGA display devices, 
etc.). In most watermarking techniques, the luminance 
25 or color values of component pixels are slightly changed 
to effect subliminal encoding of binary data through the 
image. (This encoding can be done directly in the pixel 
domain, or after the signal has t>€en processed and rep- 
resented differently - e.g. as DCT or wavelet coeffi- 
30 dents, or as conpressed data, etc.) 

[0009] While pixelated imagery is a relatively recent 
development, security documents --commonly employ- 
ing line art - go back centuries. One familiar example is 
U.S. paper cunency. On the one dollar banknote, for 
35 example, line art is used in several different ways. One 
is to form intricate webbing patterns (sometimes termed 
"guilloche patterns") around the margin of the note 
(generally comprised of light lines on dark background). 
Another is to form gray scale imagery, such as the por- 
40 trait of George Washington (generally comprised of 
dark lines on a light background). 
[0010] There are two basic ways to simulate grey- 
scales in security document line art. One is to change 
the relative soacinas of the lines to effect a lightening or 
45 darkening of an image region. Fig. 1A shows such an 
arrangement; area B looks darker than area A due to 
the closer spacings of the component lines. The other 
technique is to change the widths of the component 
lines -wider lines resulting in darker areas and nar- 
50 rower lines resulting in lighter areas. Rg. 1 B shows such 
an arrangement. Again, area B looks darker than area 
A. this time due to the greater widths of the component 
lines. These techniques are often used together Ulti- 
mately, a given region simply has more or less ink. 
55 [001 1 ] In my application 08/438.1 59 I introduced, and 
in my application 09/074.034 I elaborated on. tech- 
niques for watermarking line art by making slight 
changes to the widths, or positions, of the component 
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lines. Such techniques are further expanded in the 
present disclosure. 

[001 2] In several of my cited patents. I discussed var- 
ious "calibration signals" that can be used to facilitate 
the decoding of watermark data despite con-uption of 5 
the encoded image, such as by scaling or rotation. (See, 
e.g.. patent 5,832,119.) Common counterfeiting tech- 
niques " e.g. color photocopying, or scanning/Inkjet 
printing - often Introduce such corruption, whether 
deliberately or accidentally. Accordingly, it is important io 
that watermarks embedded in security documents be 
detectable notwithstanding such effects. Calibration sig- 
nals particularly suited for use with security documents 
are detailed in this disclosure. 

[0013] In accordance with emlxxliments of the is 
present invention, security documents are encoded to 
convey machine-readable multi-bit binary Information 
(e.g. digital watermarks), usually in a manner not alert- 
ing human viewers that such information is present. The 
documents can be provided with overt or subliminal cal- 20 
ibration patterns. When a document incorporating such 
a pattern is scanned (e.g. by a photocopier), the pattern 
facilitates detection of the encoded information notwith- 
standing possible scaling or rotation of the scan data. 
The calibration pattern can serve as a can-ier for the 25 
watermark information, or the watermark can be 
encoded independently. In one embodiment, the water- 
mark and the calibration pattern are formed on the doc- 
ument by an intaglio process, with or without ink. A 
photocopier responsive to such markings can take pre- 30 
determined action if reproduction of a security docu- 
ment is attempted. A passport processing station 
responsive to such markings can use the decoded 
binary data to access a database having information 
concerning the passport holder. Some such appara- 35 
fuses detect both the watermark data and the presence 
of a visitrfe structure characteristic of a security docu- 
ment 

[0014] The foregoing and other features and advan- 
tages of the present technology will be more readily <o 
apparent from the following detailed description, which 
proceeds with reference to the accompanying drawings. 

Brief Description of the Drawinos 

45 

[0015] 

Figs. 1A and IB show prior art techniques for 
achieving grey-scale effects using line art. 
Fig. 2 shows a virtual array of grid points that can so 
be imposed on a security document image accord- 
ing to an embodiment of the present invention. 
Fig. 3 shows a virtual an^y of regions that can be 
imposed on a security document image according 
to the Fig. 2 embodiment. ss 
Fig. 4 shows an excerpt of Fig. 3 with a line from a 
line art image passing therethrough. 
Fig. 5 shews changes to the width of the line of Fig. 



3 to effect watermark encoding. 

Fig. 6 shows changes to the position of the line of 

Fig. 3 to effect watermark encoding. 

Figs. 7A and 7B show aspects of watermark and 

calibration blocks according to an embodiment of 

the invention. 

Rg. 8 shows an illustrative reference grey-scale cal- 
ibration tile. 

Figs. 9A-9C show steps in the design of a weave 
calibration pattern according to an embodiment of 
the invention. 

Fig. 10 shows the generation of error data used in 

designing a weave calibration pattern according to 

an emtxxJiment of the invention. 

Rg. 1 1 is a block diagram of a passport processing 

station according to another embodiment of the 

invention. 

Rg. 12 is a block diagram of a photocopier accord- 
ing to another ennbodiment of the invention. 

Detailed Description 

[0016] By way of introduction, the present specifica- 
tion begins with review of techniques for embedding 
watermark data in line art. as disclosed in my applica- 
tion 09/074.034. 

[0017] Referring to Rg. 2. the earlier-described tech- 
nique employs a grid 10 of imaginary reference points 
an-ayed over a line art image. The spacing between 
points is 250 microns in the illustrated arrangement, but 
greater or lesser spacings can of course be used. 
[0018] Associated vyrith each grid point is a surround- 
ing region 12, shown in Fig. 3. As described below, the 
luminosity (or reflectance) of each of these regions 1 2 is 
slightly changed to effect subliminal encoding of binary 
data. 

[0019] Region 12 can take various shapes; the illus- 
trated rounded-rectangular shape is representative 
only, (The illustrated shape has the advantage of 
encompassing a fairly large area while introducing 
fewer visual artifacts than, e.g.. square regions.) In 
other embodiments, squares, rectangles, circles, 
ellipses, etc., can alternatively be employed. 
[0020] Rg. 4 is a magnified view of an excerpt of Rg. 
3, showing a line 14 passing through the grid of points. 
The width of the line, of course, depends on the partic- 
ular image of which it is a part. The illustrated line is 
about 40 miaons in width; greater or lesser widths can 
naturally be used. 

[0021] In one encoding technique, shown in Rg. 5, the 
width of the line is controllably varied so as to change 
the luminosity of the regions through which it passes. To 
increase the luminosity (or reflectance), the line is made 
narrower (i.e. less ink in the region). To decrease the 
luminosity, the line is made wider (i.e. more ink). 
[0022] Whether the luminance in a given region 
should be increased or decreased depends on the par- 
ticular watermarking algorithm used. Any algorithm can 
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be used, by changing the luminosity of regions 1 2 as the 
algorithm would olhenwise change the luminance or 
colors of pixels in a pixelated image. (Some watermark- 
ing algorithms effect their changes in a transformed 
domain, such as DCT, wavelet, or Fourier. However, 5 
such changes are ultimately manifested as changes in 
luminance or color.) 

[0023] In an exemplary algorithm, the binary data is 
represented as a sequence of -Is and is. instead of Os 
and 1s. (The binary data can comprise a single datum. 10 
but more typically comprises several. In an illustrative 
embodiment, the data comprises 128 bits, some of 
which are error-correcting or -detecting bits.) 
[0024] Each element of the binary data sequence is 
then multiplied by a conresponding element of a is 
pseudo-random number sequence, comprised of -1s 
and Is, to yield an intermediate data signal. Each ele- 
ment of this intermediate data signal is mapped to a cor- 
responding sub-part of the image, such as a region 12. 
(Commonly, each element is mapped to several such 20 
sub-parts.) The image in (and optionally around) this 
region is analyzed to determine its relative capability to 
conceal embedded data, and a conresponding scale 
factor is produced. Exenrplary scale factors may range 
from 0 to 3. The scale factor for the region is then multi- 25 
plied by the element of the intermediate data signal 
mapped to the region in order to yield a '*tweak" or "bias" 
value for the region. In the illustrated case, the resulting 
tweaks can range from -3 to 3. The luminosity of the 
region is then adjusted in accordance with the tweak 30 
value. A tweak value of -3 may correspond to a -5% 
change in luminosity; -2 may correspond to -2% 
change; -1 may corresporvi to -1% change; 0 may cor- 
respond to no change; 1 may correspond to + 1 % 
change; 2 may correspond to -t-2% change, and 3 may 
correspond to +5% change. (This example follows the 
basic techniques described in the Real Time Encoder 
embodiment disclosed in patent 5.71 0.834.) 
[0025] In Fig. 5, the watermarking algorithm deter- 
mined that the luminance of region A should be reduced 
by a certain percentage, while the luminance of regions 
C and D should be increased by certain percentages. 
[0026] in region A, the luminance is reduced by 
increasing the line width. In region D, the luminance is 
increased by reducing the line width; similarly in region 
C (but to a lesser extent). 

[0027] No line passes through region B, so there is no 
opportunity to change the region's luminance. This is 
not fatal to the method, however, since the exemplary 
watermarking algorithm redundantly encodes each bit 
of data in sub-pans spaced throughout the line art 
image. 

[0028] The changes to line widths in regions A and D 
of Fig. 5 are exaggerated for purposes of illustration. 
While the illustrated variance is possible, most imple- 
mentations will typically modulate the line width 3 - 50% 
(increase or decrease). 

[0029] (Many watermarking algorithms routinely oper- 



ate within a signal margin of about +/• 1% changes in 
luminosity to effect encoding. That is, the 'noise" added 
by the encoding amounts to just 1% or so of the under- 
lying signal. Lines typically don*t occupy the full area of 
a region, so a 10% change to line width may only effect 
a 1% change to region luminosity, etc. Security docu- 
ments are different from photographs in that the artwork 
generally need not convey photorealism. Thus, security 
documents can be encoded with higher energy than is 
used in watermarking photographs, provided the result 
is still aesthetically satisfactory. To illustrate, localized 
luminance changes on the order of 10% are possible in 
security documents, while such a level of watermark 
energy in photographs would generally be considered 
unacceptable. In some contexts, focalized lumin. ^ 
changes of 20, 30. 50 or even 100% are acceptable 
[0030] In the illustrated technique, the change to t: 
width is a function solely of the watermark tweak (or 
watermark/calibration pattern tweak, as discussed 
below) to be applied to a single region. Thus, if a line 
passes through any part of a region to which a tweak of 
2% is to be applied, the line width in that region is 
changed to effect the 2% luminance difference. In vari- 
ant techniques, the change in line width is a function of 
the line's position in the region. In particular, the change 
in line width is a function of the distance between the 
region's center grid point and the line's closest 
approach to that point. If the line passes though the grid 
point, the full 2% change is effected. At successively 
greater distances, successively snnaller changes are 
applied. The manner in which the magnitude of the 
tweak changes as a function of line position within the 
region can be determined by applying one of various 
interpolation algorithms, such as the bi-linear, bi-cubic, 
35 cubic splines, custom curve, etc. 

[0031] In other variant techniques, the change in line 
width in a given region is a weighted function of the 
tweaks for adjoining or sun-ounding regions. Thus, the 
line width in one region may be increased or decreased 
40 in accordance with a tweak value corresponding to one 
or more adjoining regions. 

[0032] Combinations of the foregoing techniques can 
also be employed. 

[0033] In the foregoing techniques, it is sometimes 
45 necessary to trade-off the tweak values of adjoining 
regions. For example, a line may pass along a border 
between regions, or pass through the point equidistant 
from four grid points ("equidistant zones"). In such 
cases, the line may be subject to conflicting tweak val- 
50 ues - one region may warrt to increase the line width, 
while another may want to decrease the line width. (Or 
both may want to increase the line width, but differing 
amounts.) Similarly in cases where the line does not 
pass through an equidistant zone, but the change in line 
55 width is a function of a neighborhood of regions whose 
tweaks are of different values. Again, known interpola- 
tion functions can be employed to determine the weight 
to be given the tweak from each region in determining 
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what change is to be made to the line width in any given 
region. 

[0034] In the exemplary watermarking algorithm, the 
average change in luminosity across the security docu- 
ment image is zero, so no generalized lightening or 
darkening of the image is apparent. The localized 
changes in luminosity are so^ minute in magnitude, and 
localized in position, that they are essentially invisible 
(e.g. inconspicuous/subliminal) to human viewers. 
[0035] An alternative technique is shown in Fig. 6, in 
which line position is changed rather than line width. 
[0036] In Fig. 6 the original position of the line is 
shown in dashed form, and the changed position of the 
line is shown in solid form. To decrease a region's lumi- 
nosity, the line is moved slightly closer to the center of 
the grid point; to increase a region's luminosity, the line 
Is moved slightly away. Thus, in region A, the line is 
moved towards the center grid point, while in region D it 
is moved away. 

[0037] It will be noted that the line on the left edge of 
region A does not return to its nominal (dashed) position 
as it exits the region. This is because the region to the 
left of region A also is to have decreased luminosity. 
Where possible, it is generally preferable noi to return a 
line to its nominal position, but instead to permit shifted 
lines to remain shifted as they enter adjoining regions. 
So doing permits a greater net line movement within a 
region, inaeasing the embedded signal level. 
[0038] Again, the line shifts in Fig. 6 are somewhat 
exaggerated. More typical line shifts are on the order of 
3 • 50 microns. 

[0039] One way to think of the Fig. 6 technique is to 
errploy a magnetism analogy. The grid point in the 
center of each region can be thought of as a magnet. It 
either attracts or repels lines. A tweak value of -3, for 
example, may correspond to a strong-valued attraction 
force; a tweak value of+2 may correspond to a middle- 
valued repulsion force, etc. In Fig. 6, the grid point in 
region A exhibits an attraction force (i.e. a negative 
tweak value), and the grid point in region D exhibits a 
repulsion force (e.g. a positive tweak value). 
[0040] The magnetic analogy is useful because the 
magnetic effect exerted on a line depends on the dis- 
tance between the line and the grid point. Thus, a line 
passing near a grid point is shifted more in position than 
a line near the periphery of the region. 
^041 ] (Actually, the magnetism analogy can serve as 
more than a conceptual tool. Instead, magnetic effects 
can be modeled in a computer program and serve to 
synthesize a desired placement of the lines relative to 
the grid points. Afbitrarily customized magnetic fields 
can be used.) 

[0042] Each of the variants applicable to Fig. 5 is like- 
wise applicable to Fig. 6. 

[0043] Combinations of the embodiments of Figs. 5 
and 6 can of course be used, resulting in increased 
watermark energy, better signal-to-noise ratio and, in 
many cases, less noticeable changes. 



[0044] In still a further technique, the luminance in 
each region is changed while leaving the line 
unchanged. This can be effected by sprinkling tiny dots 
of ink in the othenwise-vacant parts of the region. In high 

5 quality printing, of the type used with security docu- 
ments, droplets on the order of 3 microns in diameter 
can be deposited. (Still larger droplets are still beyond 
the perception threshold for most viewers.) Speckling a 
region with such droplets (either in a regular array, or 

JO random, or according to a desired profile such as Gaus- 
sian), can readily effect a 1% or so change in luminosity. 
(Usually dark droplets are added to a region, effecting a 
decrease in luminosity. Increases in luminosity can be 
effected by speckling with a light colored ink, or by form- 

75 ing light voids in line art othenwise present in a region.) 
(Actually, production realities often mean that many 
such microdots will not print, but statistically some will.) 
[0045] In a variant of the speckling technique, very 
thin mesh lines can be inserted in the artwork - again to 

20 slightly change the luminance of one or more regions 
(so-called "background tinting"). 
[0046] The following portion of the specification 
reviews a calibration, or synchronization pattern used in 
an illustrative security document to facilitate proper reg- 

25 istration of the watermark data for decoding. It may be 
helpful to being by reviewing further details about the 
illustrative watermarking method. 
[0047] Refen-ing to Fig. 7A. an exenrplary watermark 
is divided into "cells" that are 250 microns on a side, 

30 each conveying a single bit of information. The cells are 
grouped into a 'block" having 128 cells on a side (i.e. 
16.384 cells per block). The blocks are tiled across the 
region being watermarked (e.g. across the face of a 
security document). 

35 [0048] As noted, the watermark payload consists of 
128 bits of data. Each bit is represented by 128 different 
cells within each block. (The mapping of bits to cells can 
be pseudo-random, sequential, or otherwise.) The 128 
"0"s and "1 "s of the watermark data are randomized into 

40 substantially equal-probability "rs and "-l"s by a 
pseudo-random function to reduce watermark visibility. 
Where a cell has a value of "1," the luminance of the 
corresponding area of the image is slightly increased; 
where a cell has a value of "-1." the luminance of the 

45 corresponding area of the image is slightly decreased 
(or vice versa). In some embodiments, the localized 
changes to image luminance due to the +1/-1 water- 
mark cell values are scaled in accordance with data-hid- 
ing attributes of the local area (e.g. to a range of -«-/- 4 

50 digital numbers) to increase the robustness of the 
watermark without compromising its imperceptibility. 
[0049] It should be noted that a single watermark 
"cell" commonly encompasses a large number of ink 
droplets. In high resolution printing, as is commonly 
55 used in security documents (e.g. 5000 microdroplets 
per inch), a single watermark cell may encompass a 
region of 50 droplets by 50 droplets. In other embodi- 
ments, a cell may encompass greater or lesser numbers 
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of droplets. 

10050] Decoding a watermark requires precise re-reg- 
istration of the scanned document image, so the water- 
mark cells are located where expected. To facilitate 
such registration, a calibration signal can be employed. 
[0051] An exemplary calibration signal is a geometri- 
cal pattern having a known Fourier-Mellin transform. As 
described in patent 5.862,260. when a known pattern is 
transformed into the Fourier domain, and then further 
transformed into the Fourier-Mellin domain, the trans- 
formed data indicates the scale and rotation of the pat- 
tan. If this pattern is replicated on a security document 
that is thereafter scanned (as noted, scanning com- 
monly introduces rotation, and sometimes scaling), the 
F-M transform data indicates the scale and rotation of 
the scanned data, facilitating virtual re-registration of 
the security document image for watermark detection. 
[0052] As shown in Fig. 7B. an illustrative geometrical 
calibration pattern is a block. 3.2 cm on a side. The 
block comprises a 16 x 16 array of substantially identical 
tiles, each 2 mm on a side. Each tile, in term, comprises 
an 8 X 8 array of component cells. 
[0053] As described below, the geometrical calibration 
pattern in the illustrated embodiment is a visible design 
feature on the security document. Accordingly, unlike 
the watermark data, the calibration pattern does not 
have to be limited to a small range of digital numbers in 
order to keep it substantially hidden among other fea- 
tures of the document. Also unlike the watermark data, 
the illustrated calibration pattern is not locally scaled in 
accordance with data hiding attributes of the security 
document image. 

[0054] It is possible to print rectangular grids of grey- 
scaled ink on a document to serve as a calibration pat- 
tern. However, aesthetic considerations usually discour- 
age doing so. Preferable is to realize the calibration 
pattern in a more traditional art form, such as a seem- 
ingly random series of intertwining lines, forming a 
weave-like pattern that is printed across part or all of the 
document. 

[0055] To aeate this weave-like calibration pattern, a 
designer first defines an 8 x 8 cell reference calibration 
tile. Each cell in the tile is assigned a grey-scale value. 
In the illustrated embodiment, values within 2-10 per- 
cent ji each other are used, although this is not essen- 
tial. An exemplary reference calibration tile is shown in 
Fig. 8 (assuming 8-bit quantization). 
[0056] The Fourier-Mellin transform of a block derived 
from this reference calibration tile will serve as the key 
by which tiie scale and rotation of a scanned security 
document image are determined. 
[0057] There is some optimization that may be done 
in selecting/designing the pattern of grey-scale values 
that define the reference calibration tile. The pattern 
should have a F-M transform that is readily distin- 
guished from those of other design and watermark ele- 
ments on the security document. One design procedure 
effects a trial F-M transform of the rest of the security 



document design, and works backwards from tiiis data 
to sdect a reference calibration tile that is readily distin- 
guishable. 

[0058] Once a reference tile pattern is selected, tiie 

5 next steps iteratively define a tile having a weave-like 
pattern whose local luminance values approximately 
match the reference tile's grey-scale patten. 
10059] Referring to Rg. 9A. tiie first such step is to 
select points on the bottom and left side edges of the tile 

10 where lines are to cross the tile boundaries. The angles 
at which the lines cross these boundaries are also 
selected. (In the illustrated embodiment, these points 
and angles are selected arbitrarily, although in other 
embodiments, the choices can be made in conformance 

15 with an optimizing design procedure.) 

[0060] The selected points and angles are then repli- 
cated on the corresponding right and top edges of the 
tile. By tiiis arrangement, lines exiting tiie top of one tile 
seamlessly enter tiie bottom of the adjoining tile at the 

so same angle. Likewise, lines exiting either side of a tile 
seamlessly join with tines in the laterally adjoining 
blocks. 

[0061] The designer next establishes trial line paths 
snaking ttirough the tile (Figs. 9B. 9C). linking arbitrarily 

25 matched pairs of points on the tile's edges. (These 
snaking patiis are sometimes termed "worms.") Desira- 
bly, these paths pass tiirough each of tiie 64 component 
cells forming the tile, with the total path length through 
each cell being witfiin +1/- 30% of the average path 

30 length tiirough all cells. (This trial routing can be per- 
formed with pencil and paper, but more commonly is 
done on a computer graphics station, with a mouse, 
light pen. or otiier input device toeing manipulated by the 
designer to establish tiie routing.) In the illustiBted 

35 embodiment, tiie lines have a widtii of about 30- 100 
microns, and an average spacing between lines of 
about 100 - 400 microns, altiiough these parameters 
are not critical. 

[0062] Turning next to Fig. 10, the trial tile is assem- 
40 bled witii like tiles to form a 16 x 1 6 trial block (3.2 cm on 
a side), with a repetitive weave pattern formed by repli- 
cation of the line pattern defined on ttie 8 x 8 cell trial 
tile. This trial block is tiien convened into grey-scale val- 
ues. The conversion can be done by scanning a printed 
45 representation of the trial block, or by computer analysis 
of the line lengths and positions. The output is a 128 x 
128 array of grey-scale values, each value correspond- 
ing to the luminance of a 250 micron cell within the trial 
block, 

50 [0063] This grey-scale data is compared witii grey- 
scale data provided by assembling 256 of tiie reference 
calibration tiles (each an 8 x 8 an-ay of cells) into a 16 x 
16 calibration pattern block. In particular, tine grey-scale 
array resulting from the trial block is subtracted from the 
55 grey-seal e array resulting from the reference block, gen- 
erating a 128 X 128 array of error values. This error data 
is used to tweak the arrangement of lines in tiie trial 
block. 
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[0064] In cells of the trial calibration block where the 
error value is positive, the line is too long. That is, the 
pattern is too dark in those cells (i.e. it has a low lumi- 
nance grey-scale value), due to a surplus of line length 
(i.e. too much ink). By shortening the line length in those 
cells, their luminance Is increased (i.e. the cell is light- 
ened). Shortening can be effected by straightening 
curved arcs, or by relocating a line's entrance and exit 
points in a cell so less distance is traversed through the 
cell. 

[0065] Conversely, in cells where the error value is 
negative, the line is too short. By increasing the line 
length in such cells, their luminance is decreased (i.e. 
the cell is darkened). Increasing the line length through 
a cell can be accomplished by increasing the curvature 
of the line in the cell, ch- by relocating a line's entrance 
and exit points along the boundary of the cell, so more 
distance is traversed through the cell. 
[0066] A computer program is desirably employed to 
effect the foregoing changes in tine routing to achieve 
the desired darkening or lightening of each cell. 
[0067] After line positions in the trial calibration block 
have been tweaked in this fashion, the trial block is 
again converted to grey-scale values, and again sub- 
tracted from the reference block. Again, an array of error 
values is produced. The positions of the lines are then 
further tweaked in accordance with the error values. 
[0068] The foregoing steps of tweaking line routes in 
accordance with error signals, converting anew into 
grey-scale, and computing new error values, is repeated 
until the luminance of the resulting weave pattern in the 
trial block is arbitrarily close to the luminance of the ref- 
erence block. Four of five iterations of this procedure 
commonly suffice to converge on a final caltoration 
block. 

[0069] (It will be noted that the initial tile pattern cre- 
ated by the designer is done at the tile level -8x8 cells. 
Atter the initial trial tile is created, subsequent process- 
ing proceeds at the block level (128 x 128 cells). A com- 
mon result of the iterative design procedure is that the 
component tiles lose their uniformity. That is, the pattern 
of lines in a tile at a corner of the final calibration block 
will generally be slightly different than the pattern of 
lines in a tile near the center of the block.) 
[0070] After the final calibration block pattern has 
been established as above, the blocks are tiled repeti- 
tively over some or all of the security document, and can 
serve either as a background design element, or as a 
more apparent element of the design. By printing this 
weave pattern in an ink color close to the paper sub- 
strate color, the patterning is highly unobtrusive. (If a 
highly contrasting ink color is used, and if the pattern 
extends over most or all of the security document, it may 
be desirable to employ a brighter luminance paper than 
othenwise. since the weave pattern effectively darkens 
the substrate.) 

[0071] As noted in my application 08/649.419. ttie 
Fourier-Mellin transform has the property that the same 
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output pattern is produced, regardless of rotation or 
scaling of the input image. The invariant output pattern 
is shifted in one dimension proportional to image rota- 
tion, and shifted in another dimension proportional to 
5 image scaling. When an image whose F-M transform is 
known, is thereafter rotated and/or scaled, the degree of 
rotation and scaling can be determined by observing 
the degree of shift of the transformed F-M pattern in ttie 
two dimensions. Once ttie rotation and scale are known, 
10 reciprocal processing of the image can be performed to 
restore the image to its original orientation and scale. 
[0072] In the above-described embodiment, the cali- 
bration block pattern has a known F-M transform. When 
a security document incorporating such a pattern is 
15 scanned (e.g. by a photocopier, a flattDed scanner, a fac- 
simile machine, etc.). the resufting data can be F-M 
transformed. The known F-M pattern is then identified in 
the transformed data, and its two-dimensional shift indi- 
cates the scale and rotation conruption of the scanned 
20 security document data. With these parameters known, 
misregistration of the security document - including 
scale and rotation corruption - can be backed-off, and 
the security document data restored to proper align- 
ment and scale. In this re-registered state, the water- 
25 mark can be detected. (In alternative embodiments, the 
original scan data is not processed to remove the 
scale/rotation effects. Instead, subsequent processing 
proceeds witii the data in its corrupted state, and takes 
into account the specific corruption facta(s) to nonethe- 
30 less yield accurate decoding, etc.) 

[0073] The just-desCTibed calbration pattern and 
design procedure, of course, are just exemplary, and 
are subject to numerous modifications. The dimensions 
can be varied at will, ft is not essential ttiat the cell size 
35 Of the calibration tiles match that of ttie watermark. Nor 
do the cells sizes need to be integrally related to each 
other. Nor does the calibration pattern need to be imple- 
mented as lines; other ink patterns can alternatively be 
used to approximate the grey-scale reference pattern 
40 [0074] There is no requirement that the lines snake 
continuously through the tiles. A line can connect to just 
a single edge point of a tile, resulting in a line that 
crosses that tile boundary, but no other. Or a line can 
both begin and end in a single tile, and not connect to 
45 any other. 

[0075] While darker lines on a lighter background are 
illustrated, lighter lines on a darker background can 
alternatively be employed. 

[0076] The iterative design procedure can employ the 
50 F-M transform (or other transform). For exanple, the 
trial block pattern can be transformed to the F-M 
domain, and there compared with the F-M transform of 
the reference block. An F-M domain error signal can 
ttius be obtained, and the routing of the lines can be 
55 changed in accordance therewith. 

[0077] Afthough the illustrated embodiment tweaked 
the cell-based grey-scales of the calibration block by 
changing line curvature and position, other luminance 



13 



EP0 981 113 A2 



14 



changing techniques can be employed. For example, 
the width of the weave lines can be locally ctianged, or 
small ink dots can be introduced into certain cell areas. 
[0078] The foregoing (and following) discussions con- 
template that the watermark and/or calibration pattem is 
printed at the same time as (indeed, sometimes as part 
of) the line art on the security document. In many appli- 
cations it is desirable to provide the calibration pattern 
on the security document substrate prior to printing. The 
markings can be ink applied by the manufacturer, or can 
be embossings applied. e.g., by rollers in the paper- 
making process. (Such textural marking is discussed 
further below.) Or, the markings can be applied by the 
security document printer, as a preliminary printing 
operation, such as by offset printing. By using an ink 
color/density that is already closely matched to the 
underlying tint of the paper stock, the manufacturer of 
the paper can introduce less tinting during its manufac- 
ture. Such tinting will effectively be replaced by the pre- 
liminary printing of the watermark/calibration pattern on 
the blank paper. 

[00791 Calibration signals entirely different than those 
detailed above can also be used. Calibration signals 
that are optimized to detect rotation, but not scaling, can 
be employed when scaling is not a serious concern. 
DOT and Fourier transfornns provide data that is readily 
analyzed to determine rotation. A calibration signal can 
be tailored to stand out in a typically low-energy portion 
of the transformed spectrum (e.g. a series of fine lines 
at an inclined angle transfornns to a usually vacant 
region in DOT space), and the scanned image can be 
transformed to the DCT/Fourier domains to examine 
any shift in the calibration signal (e.g. a shift in the spa- 
tial frequency representation of the inclined lines). 
[0080] In some security documents, the just- 
described calibration weave is printed independently of 
the watermark encoding. In other embodiments, the 
weave serves as the lines whose widths, locations, etc.. 
are modulated by the watermark data, as detailed 
herein and in application 09/074,034. 
[0081 1 In an illustrative embodiment, the printing of the 
security document is achieved by intaglio printing. 
Intaglio is a well known printing process employing a 
metal plate into which the security document pattern is 
etched oi engraved. Ink is applied to the plate, filling the 
etched recesses/grooves. Paper is then pressed into 
the plate at a very high pressure (e.g. 10-20 tons), both 
raised-inking and slightly deforming (texturing) the 
paper. 

[0082] Although ink is commonly used in the intaglio 
process, it need not be in certain embodiments of the 
present invention. Instead, the paper texturing provided 
by the intaglio pressing - alone -- can suffice to convey 
watermark data. (Texturing of a medium to convey 
watemriark information is disclosed in various of my 
prior applications, including allowed application 
08/438.159.) 

[0083] To illustrate, an intaglio plate was engraved 



(using a numerically controlled engraving apparatus), to 
a depth of slightly less than 1 mm, in accordance with a 
3.2 x 3.2 cm. noise-like block of watermark data. The 
watermark data was generated as described above 

5 (e.g. 128 bits of data, randomly distributed in a 128 x 
128 cell array), and summed with a correspondingly- 
sized block of calibration data (implemented as discrete 
grey-scaled cells, rather than the line/weave pattem 
detailed alDove). In this emtjodiment, the data was not 

JO kept within a small range of digital numbers, but instead 
was railed to a full 8-bit dynamic range.) Banknote 
paper was intaglio-pressed into this plate - without ink - 
- yieWing a generally flat sut)strate with a 3.2 x 3.2 cm 
textured region therein. Only on fairly dose inspection 

15 was the texturing visible; on casual inspection the paper 
surface appeared uniform. 

[0084] This textured paper was placed - textured 
extrema down - on the platen of an conventional flatbed 
scanner (of the sort commonly sold as an accessory for 

20 personal computers), and scanned. The resulting image 
data was input to Adobe's Photoshop image processing 
software, version 4.0, which includes Diginr^rc water- 
mark reader software. The software readily detected the 
watermark from the textured paper, even when the 

25 paper was skewed on the scanner platen. 

[0085] The optical detection process by which a 
seemingly blank piece of paper can reliably convey 128 
bits of data through an inexpensive scanner has not 
been analyzed in detail; the degree of localized rellec- 

30 tion from the paper may be a function of whether the illu- 
minated region is concave or convex in shape. 
Regardless of the explanation, it is a remarkable phe- 
nomenon to witness. 

[0086] A second experiment was conducted with the 
35 same engraved plate, this time using transparent ink. 
The results were similar, although detection of the 
watermark data was not always as reliable as in the ink- 
less case. The raised transparent ink may serve as light 
conduit, dispersing the incident illumination in unpre- 
40 dictable ways as contrasted with simple reflection off 
un-inked paper. 

[0087] Experiments have also been conducted using 
traditional opaque inks. Again, the watermark can relia- 
bly be read. 

45 [0088] In addition to the just-described technique for 
"reading" intaglio markings by a conventional scanner, a 
variant technique is disclosed in Van Renesse, Optical 
Inspection Techniques for Security Instrumentation. 
SPIE Proc. Vol. 2659. pp. 159-167 (1996), and can 

50 alternatively be used in embodiments according to the 
present invention, 

[0089] Although intaglio is a prefen-ed technique for 
printing security documents, it is not the only such tech- 
nique. Other familiar techniques by which watermarks 
55 and calibration patterns can be printed include offset 
litho and letterpress, as well as Inkjet printing, xero- 
graphic printing, etc. And, as noted, textured water- 
marking can be effected as part of the paper-making 
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process, e.g. by high pressure textured rollers. 
[0090] In. still other embodiments, the watermark 
and/or calibration (Information") patterns are not 
printed on the security document substrate, but rather 
are formed on or in an auxiliary layer that is laminated 
with a base substrate. If a generally clear laminate is 
used» the information patterns can be realized with 
opaque inks, supplementing the design on the underly- 
ing substrate. Or the added information can be encoded 
in textural form. Combinations of the foregoing can sim- 
ilarly be used. 

[0091] To retrofit existing security document designs 
with information patterns, the existing artwork must be 
modified to effect the necessary additions and/or 
tweaks to localized security document luminance and/or 
texture. 

[0092] When designing new security documents, it 
would be advantageous to facilitate integration of infor- 
mation patterns into the basic design. One such 
arrangement is detailed in the following discussion. 
[0093] Many security documents are still designed 
largely by hand. A designer works at a drafting table or 
computer workstation, and spends many hours laying- 
out minute (e.g. 5 mm x 5 mm) excerpts of the design. 
To aid integration of watermark and/or calibration pat- 
tern data in this process, an accessory layout grid can 
be provided, identifying the watermark "bias" (e.g. -3 to 
+3) that is to be included in each 250 micron cell of the 
security document. If the accessory grid indrcates that 
the luminance should be slightly increased in a cell (e.g. 
1%), the designer can take this bias in mind when defin- 
ing the composition of the cell and include a touch less 
ink than might othenwise be included. Similarly, if the 
accessory grid indicates that the luminance should be 
somewhat strongly increased in a cell (e.g. 5%), the 
designer can again bear this in mind and try to include 
more ink than might othenwise be included. Due to the 
substantial redundancy of most watermark encoding 
techniques, strict compliance by the designer to these 
guidelines is not required. Even loose compliance can 
result in artwork that requires little, if any, further modifi- 
cation to reliably convey watermark and/or calibration 
information. 

[0094] Such "designing-in" of embedded information 
in security documents is facilitated by the number of 
arbitrary design choices made by security document 
designers. A few examples from U.S. banknotes include 
the curls in the presidents' hair, the drape of clothing, 
the douds in the skies, the shrubbery in the landscap- 
ing, the bricks in the pyramid, the fill patterns in the let- 
tering, and the great number of arbitrary guilloche 
patterns and other fanciful designs, etc. All include 
curves, folds, wrinkles, shadow effects, etc.. about 
which the designer has wide discretion in selecting local 
luminance, etc. Instead of making such choices arbitrar- 
ily, the designer can make these choices deliberately so 
as to serve an informational -- as well as an aesthetic -- 
function. 



[0095] To further aid the security document designer, 
data defining several different information-carrying pat- 
terns (both watermark and/or calibration pattern) can be 
stored on mass storage of a computer a workstation 

5 and serve as a library of design elements for future 
designs. The same user-interface techniques that are 
employed to pick colors in image-editing software (e.g. 
Adobe Photoshop) and fill textures in presentation pro- 
grams (e.g. Microsoft PowerPoint) can similarly be used 

10 to present a palette of information patterns to a security 
document designer. Clicking on a visual representation 
of the desired pattern makes the pattern available for 
inclusion in a security document being designed (e.g. 
filling a desired area). 

15 [0096] In the embodiment earlier-desaibed, the cali- 
bration pattern is printed as a visible artistic element of 
the security document. However, the same calibration 
effect can be provided subliminal ly if desired. That is, 
instead of generating artwork mimicking the grey-scale 

20 pattern of the reference calibration block, the reference 
calibration block can itself be encoded into the security 
document as small changes in local luminance. In many 
such embodiments, the bias to localized document 
luminance due to the calibration pattern is simply added 

25 to the bias due to the watermark data, and encoded like 
the watermark data (e.g. as localized changes to the 
width or position of component line-ad lines, as inserted 
ink droplets, etc.). 

[0097] The uses to which the 128 bits of watermark 

30 data can be put in security documents are myriad. Many 
are detailed in the materials cited above. Examples 
include postal stanps encoded with their value, or with 
the zip code of the destination to which they are 
addressed (or from which they were sent); banknotes 

35 encoded with their denomination, and their date and 
place of issuance; identification documents encoded 
with authentication infornnation by which a person's 
identify can be verified; etc.. etc. 
[0098] The encoded data can be in a raw form - avail- 

40 able to any reader having the requisite key data (in 
watermarking techniques where a key data is used), or 
can be erKrypled, such as with public key encryption 
techniques, etc. The encoded data can embody infor- 
mation directly, or can be a pointer or an index to a fur- 

45 ther collection of data In which the ultimate information 
desired is stored. 

[0099] For example, water nnark data in a passport 
need not encode a complete dossier of infornnation on 
the passport owner. Instead, the encoded data can 

50 include key data (e.g. a social security number) identify- 
ing a particular record in a remote database in which 
biographical data pertaining to the passport owner is 
stored. A passport processing station employing such 
an arrangement is shown in Fig. 11. 

55 [01 00] To decode watermark data, the security docu- 
ment must be converted into electronic image data for 
analysis. This conversion is typically performed by a 
scanner. 
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~[0101] Scanners are well known, so a detailed 
description is not provided here. Suffice it to say that 
scanners conventionally employ a line of closely spaced 
photodetector cells that produce signals related to the 
amount of the light reflected from successive swaths of 
the document, Mos\ inexpensive consumer scanners 
have a resolution of 300 dots per inch (dpi), or a center 
to center spacing of component photodetectors of about 
84 microns. Higher quality scanners of the sort found in 
most professional imaging equipment and photocopiers 
have resolutions of 600 dpi (42 microns), 1200 dpi (21 
microns), or better. 

[0102] Taking the example of 300 dpi scanner (84 
micron photodetector spacing), each 250 micron region 
12 on the security document will correspond to atxjut a 
3x3 array of photodetector samples. Naturally, only in 
rare instances will a given region be physically regis- 
tered with the scanner so that nine photodetector sam- 
ples capture the luminance in that region, and nothing 
else. More commonly, the image is rotated with respect 
to the scanner photodetectors, or is longitudinally mis- 
aligned (i.e. some photodetectors image sub-parts of 
two adjoining regions). However, since the scanner 
oversamples the regions, the luminance of each region 
can unambiguously be determined. 
[01031 In one embodiment, the scanned data from the 
document is collected in a two dimensional anay of data 
and processed to detect the embedded calibration infor- 
mation. The scanner data is then processed to effect a 
virtual re-registration of the document image. A soft- 
ware program next analyzes the statistics of the re-reg- 
istered data (using the techniques disclosed in my prior 
writings) to extract the bits of the embedded data. 
[0104] (Again, the reference to my earlier watermark 
decoding techniques is exemplary only. Once scanning 
begins and the data is available in sampled form, it is 
straightfonward to apply any other watermark decoding 
technique to extract a correspondingly-encoded water- 
mark. Some of these other techniques employ domain 
transformations (e.g. to wavelet, DCT, or Fourier 
domains, as part of the decoding process).) 
[0105] In a variant embodiment, the scanned data is 
not assemt)led in a complete array prior to processing. 
Instead, it is processed in real-time, as it is generated, in 
order to detect embedded watermark data without 
delay. (Depending on the parameters of the scanner, it 
may be necesxiv y to scan a half-inch or so of the docu- 
ment before the statistics of the resulting data unambig- 
uously indicate the presence of a watermark.) 
[0106] In other embodiments, hardware devices are 
provided with the capability to recognize embedded 
watermark data in any document images they process, 
and to respond accordingly 

[0107] One example is a color photocopier. Such 
devices employ a color scanner to generate sampled 
(pixel) data corresponding to an input media (e.g. a dol- 
lar bill). If watermark data associated with a security 
document is detected, the photocopier can take one or 



more steps. 

[0108] One option is simply to intenupt copying, and 
display a message reminding the operator that it is ille- 
gal to reproduce cun-ency. 
5 [01 09] Another option is to dial a remote service and 
report the attempted banknote reproduction. Photocop- 
iers with dial-out capabilities are known in the art (e.g. 
patent 5,305,199) and are readily adapted to this pur- 
pose. The remote service can be an independent serv- 
10 ice, or can be a government agency 

[01 1 0] Yet another option is to permit the copying, but 
to insert forensic tracer data in the resultant copy. This 
tracer data can take various forms. Steganographically 
encoded binary data is one exannple. An example is 
15 shown in patent 5,568,268. The tracer data can memo- 
rialize the serial number of the machine that made the 
copy and/or the date and time the copy was made. To 
address privacy concerns, such tracer data is not nor- 
mally inserted in all photocopied output, but is inserted 
20 only when the subject being photocopied is detected as 
being a security document. (An exanrple of such an 
arrangement is shown in Fig. 12.) 
[01 1 1 ] Desirably, the scan data is analyzed on a line- 
by-line basis in order to identify illicit photocopying with 
25 a minimum of delay. If a security document is scanned, 
one or more lines of scanner output data may be pro- 
vided to the photocopier's reprographic unit before the 
recognition decision has been made. In this case the 
photocopy will have two regions: a first region that is not 
30 tracer-marked, and a second, subsequent region in 
which the tracer data has been inserted. 
[01 1 2] Photocopiers with other means to detect not- 
to-be-copied documents are known in the art. and 
employ various response strategies. Exannples are 
35 detailed in US. Patents 5.583.614, 4.723.149. 
5,633,952, 5.640,467, and 5.424,807. 
[01 1 3] Another hardware device that can employ the 
foregoing principles is a standalone scanner. A pro- 
grammed processor (or dedicated Inardware) inside the 
40 scanner analyzes the data being generated by the 
device, and responds accordingly 
[01 14] Yet another hardware device that can employ 
the foregoing principles is a printer. A processor inside 
the device analyzes graphical image data to be printed, 
45 looking for watennarks associated with security docu- 
ments. 

[0115] For both the scanner and printer devices, 
response strategies can include disat)!ing operation, or 
inserting tracer information. (Such devices typically do 

50 not have dial-out capabilities.) 

[01 1 6] Again, it is desirable to process the scanner or 
printer data as it becomes available, so as to detect any 
security document processing with a minimum of delay 
Again, there will be some lag time before a detection 

55 decision is made. Accordingly, the scanner or printer 
output will be comprised of two pans, one without the 
tracer data, and another with the tracer data. 
[01 1 7] Many security documents already include visi- 
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ble structures that can be used as aids in banknote 
detection (e.g. the seal of the issuer and various geo- 
metrical markings). In accordance with a further aspect 
of the present invention, a security document is ana- 
lyzed by an integrated system that considers both the 
visible structures and watermark-embedded data. 
[0118] Visible security document structures can be 
sensed using known pattern recognition techniques. 
Examples of such techniques are disclosed in U.S. Pat- 
ents 5,321.773. 5.390.259. 5.533.144. 5.539.841. 
5.583.614, 5.633.952, 4.723,149, 5.692,073. and 
5,424,807 and laid-open foreign applications EP 
649,1 14 and EP 766.449. 

[0119] In photocopiers (and the like) equipped to 
detect both visible structures and watermarks trom 
security documents, the detection of either can cause 
one or more of the above-noted responses to be initi- 
ated (Fig. 12). 

[0120] Again, scanners and printers can be equipped 
with a similar capability - analyzing the data for either of 
these security document hallmarks. If either is detected, 
the software (or hardware) responds accordingly. 
[0121] Identification of security documents by water- 
mark data provides an inrportant advantage over recog- 
nition by visible structures it cannot so easily be 
defeated. A security document can be doctored (e.g. by 
white-out, scissors, or less crude techniques) to 
remo\^e/obliterate the visible structures. Such a docu- 
ment can then be freely copied on either a visible struc- 
ture-sensing photocopier or scanner/printer installation. 
The removed visible structure can then be added back 
in via a second printing/photocopying operation. If the 
printer is not equipped with security document-disabling 
capabilities, image-editing tools can be used to insert 
visible structures back into image data sets scanned 
from such doctored documents, and the complete doc- 
ument can then be freely printed. By additionally includ- 
ing embedded watermark data in the security 
document, and sensing same, such ruses will not suc- 
ceed. 

[0122] (A similar ruse is to scan a security document 
inr^age on a non-security document-sensing scanner. 
The resulting innage set can then be edited by conven- 
tional image editing tools to remove/obliterate the visible 
structures. Such a data set can then be printed - even 
on a printer/photocopier that examines such data for the 
presence of visible structures. Again, the missing visible 
structures can be inserted by a subsequent print- 
ing/photocopying operation.) 

[01 23] Desirably, the visible structure detector and the 
watermark detector are integrated together as a single 
hardware and/or software tool. This arrangement pro- 
vides various economies, e.g.. in interfacing with the 
scanner, manipulating pixel data sets for pattern recog- 
nition and watermark extraction, electronically re-regis- 
tering the image to facilitate pattern 
recognition/watermark extraction, issuing control sig- 
nals (e.g. disabling) signals to the photocopier/scanner. 



etc. 

[0124] While the foregoing apparatuses are particu- 
larly concerned with counterfeit deterrence, the embed- 
ded markings can also serve other functions. Examples 

5 include banknote processing machines that perform 
denomination sorting, counterfeit detection, and circula- 

~ tion analysis functions. (I.e., banknotes with certain 
markings may be distributed through known sources, 
and their circulation/distribution can subsequently be 

10 monitored to assist in macro-economic analyses.) 
[0125] From the foregoing, it will be recognized that 
various embodiments according to the present invention 
provide techniques for embedding multi-bit binary data 
in security documents, and provide for the reliatrfe 

15 extraction of such data even in the presence of various 
forms of corruption (e.g. scale and rotation). 
[01 26] Having described and illustrated the principles 
of my invention with reference to several illustrative 
embodiments, it will be recognized that these embodi- 

20 mertts are exemplary only and should not be taken as 
limiting the scope of my invention. Guided by the forego- 
ing teachings, it should be apparent that other water- 
marking, decoding, and ant-counterfeiting technologies 
can be substituted for, and/or combined with, the ele- 

25 merrts detailed above to yield advantageous effects. 
Other features disclosed in my eariier applications can 
similarly be employed In embodiments of the technol- 
ogy detailed herein. (Thus, I have not here belabored 
application of each of the techniques disclosed in my 

30 earlier applications -- e.g. use of neural networks for 
watermark detectors - to the present subject matter 
since same is fairfy taught by reading the present disclo- 
sure in the context of my earlier work.) 
[01 27] While the technology has been described with 

35 reference to embodiments employing regular rectangu- 
lar arrays of cells, those skilled in the art will recognize 
that other an-ays -neither rectangular nor regular - can 
alternatively be used. 

[0128] While the embodiments have described the 

40 calibration patterns as adjuncts to digital watermarks - 
facilitating their detection, such patterns have utility 
apart from digital watermarks. One exanple is in re-reg- 
istering scanned security document image data to facil- 
itate detection of vislt^le structures (e.g. detection of the 

45 printed seal of the Issuer, using known pattern recogni- 
tion techniques). Indeed, the use of such calibration pat- 
terns to register both watermark and:visible structure 
image data for recognition is an important economy that 
can be gained by integration a visible structure detector 

so and a waternr^rk detector Into a single system. 

[0129] Although security documents have most com- 
monly been printed (e.g. cotton/linen), other substrates 
are gaining in popularity (e.g. synthetics, such as poly- 
mers) and are well (or better) suited for use with the 

55 above-described techniques. 

[0130] The embodiments detailed above can be 
implemented in dedicated hardware (e.g. ASICs), pro- 
grammable hardware, and/or software. 
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[OTST] Several patents and patent publications were 
noted above. The artisan is presumed to be familiar with 
the technologies described therein, and to be capable of 
Integrating same with the elements partiojlarly detailed 
in this disclosure. To the extent possible, the disclosures 
of those documents is incorporated herein by reference. 
[0132] In view of the many possible embodiments to 
which the principles of the above-descrtoed technology, 
may be put it should be recognized that the detailed 
embodiments are illustrative only and should not be 
taken as limiting the scope of my invention. Rather, I 
claim as my invention all such emtxxjiments as may 
come within the scope and spirit of the following claims 
and equivalents thereto. 

Claims 



zones of the calibration pattern. 

11, The document of claim 1 in which the calibration 
pattern comprises a tiled pattern, each tile including 

5 a line following an apparently random route. 

12. A method of producing a security document, com- 
prising: 

1 0 providing nrujlti-bit binary data; 

forming a pattern of recesses in a metal mem- 
ber, said pattern having the nrulti-bit binary 
data encoded therein; and 
pressing a substrate into said metal member to 

IS texture the substrate in accordance with said 

binary data. 



1 . A security document including at least a substrate, 
a digital watermark, and a calibration pattern, the 
calibration pattern having a known transform facili- 
tating decoding of the digital watermark from scan 
data corresponding to said document. 

2. The document of daim 1 in which the calibration 
pattern has a known Fourier-Mellin transform facili- 
tating decoding of the digital watermark from scan 
data corresponding to said document. 

3. A negotiable financial ir^rument in accordance 
with claim 1 . 

4. A passport in accordance with daim 1 . 

5. The document of daim 1 in which the calibration 
pattern has a visible weave-tike appearance. 

6. The document of daim 1 in which the calibration 
pattern extends across most of the security docu- 
ment. 

7. The document of claim 1 in which at least one of the 
digital watermark and calibration pattern are real- 
ized by texturing of the document, and not printing. 

8. The document of claim 1 in which both the digital 
watermark and the calibration pattern are realized 
by texturing of f r: Jocument, and not printing. 

9. The document of claim 1 in which each one square 
millimeter zone of the calibration pattern has an 8- 
bit grey-scale value within twenty percent of the 
value of all adjoining one square millimeter zones of 
the calibration pattern. 



13. The method of claim 12 further induding inking the 
recesses prior to the pressing. 

20 

14. The method of daim 12 in which the pattern has the 
nrtulti-bit binary data steganographically encoded 
therein, wherein a human viewer of the texture on 
the substrate is not alerted that it conveys multi-bit 

25 binary data. 

1 5. A method of processing a security document, com- 
prising: 

30 generating scan data corresponding to said 

document; 

performing a domain transformation on the 
scan data to obtain corruption data related 
thereto; and 

35 detecting embedded information from the proc- 

essed scan data, said detecting enploying said 
corruption data. 

16. The method of claim 15 in which: 

40 

the domain transformation Is a Fourier-Mellin 
transform: 

the corruption data is a scale or rotation factor; 
and 

45 the detecting indudes compensating the scan 

data in accordance with the corruption data. 

17. The method of claim 15 in which the embedded 
information comprises multi-bit digtal data. 

so 

18. The method of claim 15 which further includes 
using the detected information to access additional 
information in a database. 



10. The document of claim 1 in which each 250 micron 
by 250 micron zone of the calibration pattern has an 
8-bit grey-scale value within twenty percent of the 
value of all adjoining 250 micron by 250 micron 



55 19. Apparatus for use with security documents, com- 
prising: 

a scanner for scanning a security document 
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and producing scan data corresponding 
thereto; 

a processor responsive to encoding on the 
security document for determining rotation of 
the scan data from a reference state and pro- s 
ducing output data conesponding thereto; 
a visible structure detector ; and - 
a steganographic watermark detector; 
said output data permitting the detectors to 
detect visible structures and steganographic to 
watermarks associated with the security docu- 
ment notwithstanding at least certain rotations 
of the scan data. 
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FIG. 1 A (Prior Art) 
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FIG. 3 
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FIG. 8 
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(54) Digital watermarking and methods for security documents 



(57) Security documents (e.g. passports, currency, 
event tickets, and the like) are encoded to convey 
machine-readable multi-btt binary information (e.g. dig- 
ital watermark), usually in a manner not alerting human 
viewers that such infomnation is present. The docu- 
ments can be provided with overt or subliminal calibra- 
tion patterns. When a document incorporating such a 
pattern is scanned (e.g. by a photocopier), the pattern 
facilitates detection of the encoded infomnation notwith- 
standing possible sealing or rotation of the scan data. 
The calibration pattern can serve as a carrier for JJie 
watermark information, or the watermark can be 
encoded independently. In one embodiment, the water- 
mark and the calibration pattern are formed on the doc- 
ument by an intaglio process, with or without ink. A 
photocopier responsive to such markings can take pre- 
detennined action if reproduction of a security docu- 
ment is attempted. A passport processing station 
responsive to such markings can use the decoded 
binary data to access a database having information 
concerning the passport holder. Some such appara- 
tuses detect both the watermark data and the presence 
of a visible structure characteristic of a security docu- 
ment (e.g., a printed seat of the document's issuer). 
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